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“Not everything that counts is countable, and not everything that's countable counts.” A. Einstein (Bommann 
and Daniel 2009) 

“It is a fact that taxonomic monographs are rarely read by general biologists; in fact,many of them are read by 
only a few specialists.” (Mayr, Linsley & Usinger 1953) 

The number of citations is not an adequate measure of taxonomic quality, which is a view that is shared with other 
scientific disciplines (Seglenl997; Valdecasas et al. 2000; Walter et al. 2003). A recent editorial in Nature claims that “... 
citations are an unreliable measure of importance" (Anon. 2010: 850) and uses two chemistry papers as an example. The 
first paper was cited 182 times in the same period that the other paper was cited only 13 times. However, the latter paper 
is recognized as ‘outstanding’ by the experts from the American Chemical Society, which would not be revealed by the 
arbitrary counting of citations. Similar examples have evoked critical editorial comments in high-impact journals. In the 
context of a possible future use of the Impact Factor (IF), the editor of EMBO Reports states: 

"[The] Impact Factor, which began as an attempt to identify and recognize the academic quality of scientific 
work, is already a commercial indicator in many ways on a par with the Global Dow Jones Index... Someone 
should perhaps have done something 40 years ago to arrest the trend towards judging the value of scientific 
work by a self-inflating and, to a large extent, arbitrary number; we all know that a scientific finding’s true 
value to humanity can only be judged by history. "(Jacobs 2009: 1067). 

Mark Patterson, the Director of Publishing at PLoS, agrees with Jacobs’ assessment: 

“But what is to be done when the journal impact factor is so tightly woven into the fabric of research 
assessment? How could we escape Jacobs’s nightmare scenario? At PLoS, we believe articles should be 
judged on their own merits, rather than on the basis of the journal in which they happen to be published” 
(Patterson 2009a: 1186). 

What is striking here is how Patterson’s and Jacobs’ comments parallel Van Valen’s motto, "The primacy of content over 
display". 

Citations, IF and Taxonomy 

With the objective of addressing the 'quality of science', the first proposal to use citation indexes for science was 
published by Garfield (1955): "...I propose a bibliographic system for science literature that can eliminate the uncritical 
citation of fraudulent, incomplete, or obsolete data..." The use of journal citations in articles to specifically select those 
with a higher number of citations to add to a library was initiated by Gross & Gross in 1927 (Archambault & Lariviere 
2009). Archambault and Lariviere tracked the genesis of the practice of using citations as selection criteria to produce 
lists of'relevant journals', along with the successive incarnations and refinements of journal 'selection' procedures. This 
practice continued until Martyn & Gilchrist (1968) and then Garfield (1972) established the IF as it is currently known 
(in Archambault & Lariviere, 2009). Relevance and quality are considered strongly correlated terms by Garfield (2003) 
and others. Recently, Krell (2010) introduced the term 'attention' as an interpretation of what a citation really means. 
However, he is currently alone in assigning this particular meaning to the IF. 

Valdecasas et al. (2000) suggested that the “reliance on citation index was detrimental to taxonomy”. Our basic 
argument in that article was that the citation and journal impact factor were being used by managers and funding 
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agencies as criteria to promote and finance research. Because taxonomy is a discipline that is not highly cited, the 
professionals of taxonomy are not very competitive on the basis of these criteria. This statement has been confirmed by 
the recent trends in academia (Meier & Lim, 2009, page: 93).Subsequently, Krell (2000) proposed that IF are not relevant 
to taxonomy (and we agree with him). Garfield (2000) answered Krell (2000) and stated that "[t]axonomy is small, but it 
has its citation classics", meaning that there were taxonomic papers with a substantial number of citations. However, 
Garfield’s comments do not consider taxonomic papers that have not been cited despite their valuable contributions to 
the field. In his reply, Garfield only mentioned Robert R. Sokal and Edward O. Wilson as two representative taxonomic 
authors with citation classics. On one hand, Sokal never wrote a proper taxonomic paper, i.e., a paper that discovered, 
described or revised a taxon. Sokal is a mathematical methodologist, who occasionally worked with taxonomists (e.g., 
the bee specialist Charles D. Michener). E. O. Wilson has citation classics, but they are not his publications on taxonomy. 
The following are the publications by Wilson that are considered citation classics (none of them are strictly taxonomic 
papers; http://garfield.library.upenn.edu/classics/classics_s.html, accessed 23 May 2011): 

- Brown, W. L. and E. 0. Wilson. 1956. Character displacement. Systematic Zoology, 5: 49-64. 

- Mac Arthur, R. H. and E. 0. Wilson. 1967. The Theory of Island Biogeography. Princeton Univ. Press, Princeton, 
NJ. 203 pp. 

- Simberloff, D. S. and E. 0. Wilson. 1969. Experimental zoogeography of islands: the colonization of empty islands. 
Ecology, 50(2): 278-296. 

- Wilson, E. 0. 1971. The Insect Societies. Belknap Press of Harvard Univ. Press, Cambridge, MA. 548 pp. 

- Wilson, E. 0. 1975. Sociobiology: The New Synthesis. Belknap Press of Harvard Univ. Press, Cambridge, MA. 697 

pp. 

E.O. Wilson has more than 370 publications (see his CV at http://www.discoverlife.org/who/CV/ 
Wilson,_Edward.html, accessed 23 May 2011). In total, 231 publications were covered by the Zoological Record (ZR), 
and 260 publications were covered by the Web of Knowledge (WOK) database, which maintains a record of the number 
of citations a given paper has received. Of his taxonomic papers (i.e., papers with descriptions of species; near 40 
publications) four are included in the WOK database. These taxonomic papers in the WOK database have 22 citations in 
total. His last description of an ant species in Colombia (Fernandez & Wilson 2008; included in the WOK database), 
which was performed in 2008, has zero citations, and it is not expected to obtain more in the near future. There is no 
reason to believe that Wilson has not invested the same high quality effort in this publication as he has in any of his other 
publications. Wilson has a publication in Biotropica on tree ants that has 102 citations. However, the work is a list of 
species, which have been identified to genus or are potentially new species, without any descriptions or taxonomic notes. 
Finally, Wilson’s important paper on Mesozoic ants (Wilson et al. 1967a) that was published in the journal Psyche is not 
included in the WOK database. However, its simultaneous brief note in Science (Wilson et al. 1967b) received 33 
citations (the interesting facts that are related to taxonomy are located in the publication in Psyche, to which the 
publication in Science refers for further information). We maintain, on the basis of our personal experience, that 
managers and funding agencies rely on citations and the IF to provide financing and to make decisions about promotions. 
We now have a plethora of indexes that are based on citations, which are taken as correlates of academic quality. 
Moreover, the open portal 'Publish or Perish' (http://www.harzing.com, accessed 23 May 2011) calculates, at no cost 
(using the Google Scholar database), a set of indexes that are based on the citations of any science or humanities 
researcher. The Scimago portal (http://www.scimagojr.com/) provides an alternative journal rank to that of Thomson and 
Reuters (IF owners). The increase in the number of available indexes suggests the popularity of'citation' metrics with the 
educated public. 


Finding a way out 

There are plausible solutions to Jacobs’ ‘nightmare scenario’. Some journals are studying different methods to evaluate 
the importance of published scientific papers. The PLoS organization has its own 'article-level metrics program' that is 
based on 'usage data' and is open to external contributions to improve the program (Patterson 2009b). Walter et al. (2003) 
proposed a method based on content for medical specialties. In taxonomy, criteria such as the number of citations or 
usage data will not highlight the quality of a published paper or monograph. The current citation metrics do not reflect 
the quality of taxonomic work because the breadth of the subject matter is limited (i.e., single taxa) and taxonomists 
usually work in small cliques (Ebach et al. 2011). To find a viable citation metric for taxonomy, there is a temptation to 
develop an ‘integrative’ or ‘holistic’ taxonomic index, i.e., one that values the quality of a taxonomic paper on the basis 
of the data and the range of techniques used, regardless of their diagnostic relevance, or the use of several types of 
output, such as images or DNA (see Page 2009). In this case, taxonomists with vast morphological or DNA datasets and 
accompanying SEM micrographs would fare better than single diagnostic works. This metric would favor a taxonomist 
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from a well-funded and well-staffed lab with access to infrastructure and technology over a single museum taxonomist 
who is armed with only an old Zeiss microscope and a drawing tube.A fair metric should award good taxonomy over 
technology—the ‘primacy of content over display’. Although a fair metric may appear to be biased against the recent 
bioinformatics boom, we propose that it should not be construed as such. Bioinformatics has its place, as does high- 
quality taxonomy. Whether we use pen and paper or relational databases should not matter. The IF is clearly unfair to 
taxonomists, who primarily publish in journals with low impact factors or in journals that do not have an IF (because 
they are not covered by the WOK database). A new honest and transparent method is required that reflects the 
importance and relevance of taxonomic descriptions and systematic revisions. The following suggestions are intended as 
starting points from which the taxonomic and scientific community at large can create a new, fair and valid metric that 
assesses the taxonomic quality of published works. 


The T-index: Proposing a New Citation Metric 

I propose the following tentative Taxonomy Index (herein T-Index) as a standard metric to assess the taxonomic quality 
of a published taxonomic work. The T-index is based on the following criteria: 

1. The description and diagnosis are in accordance with the relevant code of nomenclature. 

2. The standard of description is adequate to contemporary practice in the corresponding clade. 

3. The journal has unbiased editor and referee systems. 

4. Any molecular or morphometric evidence (i.e., molecular tree) that supports a new classification is accompanied by 
a written description and a diagnosis that is in accordance to the relevant code of nomenclature. 

5. New taxa hypotheses and full revision work are accorded similar merit. 

6. Taxa diversity 'context' (see below) affects the weight assigned to criterion (5). 

The discovery of a new taxon is made in the context of another related taxon. For instance, new species are 
diagnosed within a species complex or a genus, following the various codes of nomenclature. At least in eukaryotes, a 
good taxonomic paper should provide enough data to diagnose a new taxon within a taxon on the basis of their 
morphological characteristics. However, any additional information, including cytogenetic, behavior or DNA phenotype, 
is welcome. A single species description that is based solely on the species diagnostic characteristics but is lacking a 
comparison to other members of its group is of low taxonomic value (i.e., the diagnosis may be synonymous with 
another described species). Another example includes a situation where five new species are described in a monographic 
revision of a genus consisting of 100 known species or a work consisting of a genus with 20 known species. The former 
work requires a far greater scope and knowledge than the latter. The T-index does not necessarily apply only to those 
discovering new taxa. Taxonomic revisions may uncover new discoveries and classifications, which are developed by 
new data or a new insight into the relationships between taxa (e.g., cladograms). Full taxon revisions, such as new taxon 
descriptions, will be assessed equally under the T-index. In this sense, any nomenclatural act except those that are based 
on misprints or any other trivial error will be considered. For instance, the genus Arrenurus (Acari, Hydrachnidia) has 
more than 400 described species (Viets 1987) within several subgenera and ‘species groups’. If a publication revises or 
describes a certain number of species in one of the subgenera, e.g., Truncaturus, the work will be evaluated in relation to 
the number of species that are already described in this subgenus. Any new or revised taxon description will be evaluated 
in relation to the number of taxa in its current clade, i.e., the new genus in the family clade and the new species in the 
genus clade. 


How the T-Index works 


The T-Index is measured by including only the following two variables: N,, which is the number of new taxa discovered 
or revised in the clade that is being studied with rank i (e.g., order, family, etc.), and S„ which is the number of taxa in 
that clade and rank after the study. The second variable is used to calculate a weighting factor that qualifies the effort of 
the taxon that is discovered or treated in the publication in relation to all known taxa in the clade. 
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For instance, if the study includes taxa of rank order, family, genus and species, the index would include a term for 
each of them: 


T = 


N 0 N f N g 
N 0 -^+ N f ^+ N gl f-+ N s 


Ns 

S s 


As a reference work, Table 1 tabulates the number of taxa that were treated in Darwin's four volumes on extant and 
fossil Cirripedia (the figures in Table 1 are close approximations of the total number of the nomenclatural acts by Darwin 
in 1851 and 1854 works). 


T vol j = 178.5; T vo12 = 82.5; T vol 3 = 347; T ol 4 = 85 

Monographies and revision works, such as Darwin's monograph on the Cirripedia, usually get high values when 
compared with single or few species description works, which seems reasonable if we take into account the amount of 
effort and time necessary to perform them (eight years in the case of Darwin's Cirripedia). 

Table 2 tabulates the number of new species that were found or revised in an imaginary work on the clade 
Hydrachnidia. In this case, the value of the index is very small: 


i 4 9 

248 + 126 + 183 


0.08 


The T-lndex increases as the number of taxa that are studied or revised reaches the number that is already known. In 
this example, the imaginary authors studied 0.4% of the species of the known Atractides (Atractides), 1.6% of the species 
of Monatractides and 1.6% of the species of Torrenticola (no subgenus specified). 


TABLE 1. Taxa treated in Darwin's four volumes on recent and fossil Cirripedia. 


Taxa 

Order 

Family 

Genera 

Species 

Darwin Cirripedia Vol. 1 F. 
Lepadidae 

1/3 

1/5 

40/40 

138/138 

Darwin Cirripedia Vol. 2 Fossil 
Lepadidae 

1/3 

1/5 

12/12 

70/70 

Darwin Cirripedia Vol. 3 F. 
Balanidae; F. Verrucidae 

3/3 

3/3 

41/41 

304/304 

Darwin Cirripedia Vol. 4 Fossil F. 
Balanidae and F. Verrucidae 

- 

3/3 

24/24 

60/60 


TABLE 2. Species treated in an imaginary work in the clade Hydrachnidia (Acari, Parasitengona). 


Taxa 

New and revised 

Previously known 

Total 

Atractides 

1 

247 

248 

Monatractides 

2 

124 

126 

Torrenticola 

3 

180 

183 


The T-Index in the world of academia 

One may argue that evaluating taxonomic publications is difficult to implement automatically. We propose that the 
evaluation of taxonomic publications is not necessarily difficult. For example, the Zoological Record database (which 
maintains a registry of all zoological literature that is relevant for biodiversity studies) distinguishes between new 
species, new rediagnoses and new records under their 'systematic data' field entry. In this case, it would be feasible to 
automatically extract the number of new taxa and the number of rewritten descriptions that are required to calculate the 
value of the work under examination.In addition, several sites are available on the internet to extract the number of 
species that are already known within a specific taxon. In this example, we used an open access data list for Acari (http:/ 
/insects.tamu.edu/research/collection/hallan/Acari/Family/Actinedidal.htm; accessed 24 May 2011). However, similar 
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lists for all types of organisms may be found, such as the Catalogue of Life (http://www.catalogueoflife.org/), which at 
the time of this writing had not yet completed the Acari record.The proposed index is radically different from those that 
have been previously suggested, because the IF or the H-index are based on number of citations and the T-Index is based 
on content. It may be argued that some 'bad' taxonomist may publish poor-quality papers. The T-index, like any metric, 
only gives us an indication of what is assumed to be of high quality based on the filters of the journal editors and the peer 
review process, which are responsible for assessing whether a manuscript fits the standard of its time. As the sociologist 
of science, Derek J. De la Solla Price, made explicit some time ago (De la Solla Price 1969), the publication behavior of 
science and technology are different. Academia is clearly papyrocentric, and industry is plainly papyrofobic 1 . The basic 
surrogate criteria used to value the merit of scientific work are derived from the public creation of science and are based 
on the number of citations and the IF. However, as argued above, the number of citations is a poor indication of scientific 
merit. Let us keep the honorable academic open-publishing tradition and value content rather than any other surrogate 
variable.The proposed index could be used alongside previous indices that are based on the number of citations. This 
combined approach would make it possible to gauge the resonance of a particular work in the scientific community 
without losing sight of its quality. We hope that this proposal will create a forum for debate about defining a new metric to 
assess the quality of taxonomic papers. The Science Citation Index does not adequately consider taxonomists and 
taxonomy. Any metric that works in the interest of taxonomists would present a better and fairer standard. 
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